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ABSTRACT 

This paper reports the results of an SAT I field 
trial which examined test performance and efficiency of test timing 
for students with disabilities, A prototype SAT I test was 
administered to 1,113 students with disabilities in December 1992, in 
special formats and using special accommodations, to obtain 
information about the level of performance and amount of time used by 
students with hearing, learning, visual, and/or physical 
disabilities. The test was available in braille, large type, regular 
type, cassette tape, and reader's script versions. Students taking 
regular-type and large-type versions were given 1.5 times the 
standard amount of time for each section; other formats were given at 
least double the standard time per section. The great majority of 
students were able to complete the entire test sections, irrespective 
of test format. Math sections of the test displayed wider variation 
in completion rates by test format and by test section than did 
verbal sections. Verbal scores were lowest for students taking the 
script version and highest for those taking the regular type version. 
Math scores were lowest for students using the b/aille version and 
highest for those taking the regular type version. Results are 
compared to results of a spring field trial. (JDD) 
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This past March ^ the College Board introduced the SAT I: Reasoning 
Test (SAT I) for the national testing program as well as for students 
with disabilities. Among the changes in the test are different section 
configurations; longer reading passages with an emphasis on critical 
reading skills; a new question type in the mathematical section for 
which students must supply an answer rather than choose among options; 
and optional use of a calculator on the math portion of the test. The 
purpose of this paper is to report the results of an SAT I field trial 
which examined test performance and efficiency of test timing for 
students with disabilities. 

Over the past five years a number of field trials have been carried 
out to investigate the various technical, operational, and policy issues 
related to the development of the SAT !• A large field trial was held 
in the spring of 1992 involving over 180,000 students from about 2,400 
schools. A portion of that field iirail focused on a prototype of the 
SAT I administered under standard timing conditions. The results of 
that and earlier field trials have been used to develop content and 
statistical specifications, estimate test speededness, and examine 
reliability, validity, and differential item functioning. 

As part of the commitment to assure fair testing of examinees with 
disabilities, in December 1992 Educational Testing Service (ETS) and the 
College Board conducted special field trial of the SAT I for students 
with disabilities. The prototype SAT I from the spring 1992 field trial 
was administered in special form,ats (e.g., large-type, braille) and 
using special accommodations (e.g., recorder/ scribe ) . The purpose of 



the field trial, as stated in a letter inviting high schools to 
participate, was "to obtain information about the level of performance 
and amount of time used by students with hearing, learning, visual, 
and/or physical disabilities." Another purpose was to solicit feedback 
from students, teachers, and counselors about changes in the new tests 
and the accommodations and formats used in the field trial. 

DATA COLLECTION 

High School JRecruitment 

With the help of the ETS Committee on People with Disabilities, 
schools with programs for students with specific disabilities were 
identified and invited to participate in the field trial. In addition, 
high schools that had tested students with disabilities in the past year 
were invited to participate. 

As an incentive to ■ participate, schools received student score 
rosters containing estimated SAT scores and national percentiles. 
Interpretive information was also provided* Following the field trial, 
each participating school also received copies of the SAT I in the 
various test formats along with scoring instructions. Students could 
use these practice materials to prepare for the SAT I. It was 
emphasized that scores from the field trial could not be used for 
college admissions. . 

Schools were asked to test high school juniors and seniors with 
documented disabilities, that is, students who met the eligibility 
requirements to take the operational SAT under nonstandard conditions. 
In order to ensure that all disability groups were represented in the 
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field tri?il sample, schools were asked to identify the number of 
students by disability group: blind/visually impaired, deaf/hard of 
hearing, learning disabled, physically disabled, or 
psycho logical /emotionally disabled. 

A few weeks before the field trial, all participating high schools 
were sent a test preparation leaflet containing test directions and 
sample questions • The leaflet was used to familiarize students with the 
test directions and question types on the SAT I. 
Administration 

The field trail was held during December. Schools were allowed to 
schedule the test on any convenient day during that period. (Because of 
school closings, a few schools continued testing until early January,) 
Both group and individual administrations were allowed, although group 
administrations were encouraged. Students were allowed to test on two 
consecutive days if needed. Total testing time ranged from 3 hours, 50 
minutes to 5 hours, 45 minutes, depending upon the test format 
administered. An additional 30 minutes were needed for distributing 
materials, reading instructions, and completing the questions on the 
answer sheet. 

In addition to the test questions, examinees were asked basic 
demographic questions on the answer sheet. A separate Student Feedback 
Survey was administered within two days of the test while the student's 
memory of the testing situation was still fresh. The survey contained 
questions about test preparation, perception of adequacy of test timing, 
accessibility and use of calculators, type of disability, and type of 
format or accommodations used. Results of the survey are described by 
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Wendler and Wright (1994). 
Testing Volujne 

Initially 118 high schools with around 2,000 eligible students 
expressed interest in the field trial. A total of 100 schools testing 
1,113 students returned answer sheets in time to be considered for the 
analysis groups. (One school returned answer sheets too late to be 
included in the timing analyses, but was included in the survey 
analyses.) 

DESIGN OF THE FIELD TRIAL 

Accommodations 

The test, consisting of three verbal and three matheiuatical 
sections, was available in braille, large-type, regular-type, cassette 
tape, and reader's script versions. Students using the cassette, 
braille, or script versions could make use of braille diagrams for the 
math figures. Students using the cassette version also used a printed 
version of the test (braille, large-type, or regular-type) . Students 
coded their responses on machine-scannable answer sheets or large-block 
answer sheets; the use of a typewriter and recorder/ scribe was also 
peirmitted, A sign language interpreter could be used for the spoken 
test directions. Students were allowed to use a calculator on the 
mathematical sections of the test. Other accommodations (e.g., 
magnifying glass, braille scratch paper) were also permitted. 
Timing 

No timing limits are given to students testing under SAT Services 
for Students with Disabilities. For the field trial, however, each 
section of the SAT I specified a maximum amount of time allowed. The 
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specific amount of time varied by test format. Students taking regular- 
type and large-type versions were given time and a half the standard 
time per section; other formats were given at least double the standard 
time per section* 

RESULTS 

Analysis Sample Screening 

Originally, only college-bound juniors and seniors were to be 
included in the analyses. However, small sample sizes and a high 
proportion of students who did not indicate their grade level or who 
were in 10th grade led to the decision to include all available cases. 
Students who did not answer at least one question in a verbal or math 
section were excluded from the analysis of that portion of the test. A 
total of 1,113 students from 100 schools were included in the timing 
analysis. Table 2 presents demographic characteristics cf these 
students . 
Test Performance 

Average verbal and math scores were calculated by test format. 
Since the disability of the student could not be determined from the 
answer sheet, level of performance by disability group could not be 
determined during this phase of the analysis. Test performance by 
disability group is provided in Wendler and Wright (1994) . 

Table 3 provides average scores by test format. Test performance 
for students participating in the spring 1992 field trial are included 
as a comparison group. All scores, both verbal and math, were lower for 
the group of students participating in the December field trial compared 
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to spring field trial participants. Verbal scores were lowest for those 
students taking the script version (mean = 292) of the test and highest 
for those taking the regular-type version (maan = 315) of the test. 
Math scores were lowest for students using the braille version (mean = 
308) of the test and again highest for those taking the regular-type 
version (mean 347) . 
Timing Analyses 

Several indicators were used to assess speededness as part of the 
field trial: (1) percent of students completing test section, (2) 
percent of students completing 75% of the questions in test section, and 
(3) number of questions reached by 80% of students. Questions are 
considered "not reached" when a student makes no further marks on the 
answer sheet. Information on completion rates are found in Tables 4 
through 7. 

For the total group of students, speededness data look similar in 
both the spring field trial and the December field trial. Overall, a 
higher percentage of students in the December field trial completed each 
section of the test compared to the spring field trial. The only 
exception was the Math 2 section, where a slightly smaller percentage of 
December field trial students completed the section. However, students 
participating in the spring field trial tended to complete 75% of the 
questions in a test section at a higher rate than those in the December 
field trial. For the verbal sections of the test, 80% of the students 
with disabilities reached all questions in the section. For the 
mathematical sections of the test, 80% of these students reached almost 
all questions in a particular section* In the Math 1 section, 80% of 
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the students reached all but one question; in the Math 2 section, 80% of 
the students reached all but two questions. 

Differences in completion rates are obvious across the different 
formats of the test. Most students appeared to have adequate time to 
complete the verbal sections of the test, irrespective of the test 
format. It should be noted, however, that sample sizes for braille, 
large-type, and script formats were very small and conclusions drawn 
from these groups must be treated as tentative. Cassette tape users 
showed the least amount of test speededness; number of students 
completing the verbal sections ranged from 80% to 98%. Large-type users 
showed the most test speededness; number of students completing the 
verbal sections ranged from 77% to 82%. While 80% of the students 
taking the regular-type, cassette tape, and script versions of the test 
reached all questions in the verbal sections, students taking braille or 
large-type versions of the test did not reach all questions. 

Overall, the math sections of the test seemed to be more speeded 
thai> the verbal sections of the test for all formats. For verbal, 
completion rates tended to be in the 80% to 96% range; for math, 
completion rates tended to be in the 40% to 70% range. 

The math sections of the test displayed wider variation in 
completion rates by test fomaat and by test section. All test sections 
appeared to be somewhat more speeded for students taking the large-type 
and script versions of the test than for other students. For example, 
48% of the students using large-type and 41% of the students using the 
reader's script completed the Math 1 section, compared to 67% of the 
students using regular-type and 83% using cassette. The Math 3 section 
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seemed to be less speeded for all students, regardless of the format of 
the test they used, compared to the other two math sections- For Math 
3, 80% of the students across all formats reached all cpaestions. 

DISCUSSION 

While some aspects of the SAT I field trial for students with 
disabilities mimicked the ways that students traditionally take the SAT 
under SAT Program for Students with Disabilities, other aspects of the 
field trial were somewhat "artificial." For example, students were 
allowed to use all accommodations they may use during a real test 
administration, such as alternate test formats, use of recorder/ scribe, 
and use of typewriters and other specialized equipment. While students 
taking the SAT are allowed extended time, timing restrictions by section 
are not imposed on examinees. 

Several things become apparent. First, the students participating 
in the December field trial appear to be young. Many were sophomores; 
many did not indicate their grade level, perhaps because schools were 
asked to test juniors and seniors. Second, a higher proportion of 
students left entire test sections blank than what was seen in the 
spring trial. Finally, verbal and math performance is lower for the 
students involved in the December field trial compared to those involved 
in the spring field trial. Lower performance may be attributed to 
students being younger in the December study, the nvimber of test 
sections left blank, the interplay of disabilities with testing, 
insuf f i.-rient section timing, lack of motivation on student's part, or 
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any combination of factors. 

Nevertheless, soxne information about restrictive timing on an SAT 
can be derived from the field trial. For example, even with section 
timing being imposed, the great majority of students were able to 
complete the entire test section. This can be used to guide students 
and counselors in determining the amount of time a student might need 
when taking an individual administration of the SAT I. In addition, it 
is apparent that some test formats require more time per section than 
other formats. The act of running a cassette tape and rewinding it 
through a reading passage takes more time than simply skimming through 
a passage on a regular-type test book. Students using large ^type 
formats present a special challenge. Although it is a printed media 
they use which does not reqpaire special time for "rewinding," students 
who are likely to use a large-type test appear to have disabilities 
whose nature requires more visual processing time. Using the same 
testing limits as used with regular-type students seemed to be somewhat 
inappropriate for large-type students. However, it should be noted that 
large-type sample sizes were very small. 

This paper reported on issues related to timing on the SAT I for 
students participating in the December field trial. While it does not 
provide a definitive answer to what appropriate timing is for students 
with disabilities, it does provide a framework for further investigating 
the issue. As the SAT I is introduced, additional data relevant to 
section timing and total test timing will be routinely collected and 
analyzed. 
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Table 1 

Timing Liinits by Section and Test Fonaat 



Test 
Section 


Standard 


Field Trial Timina 


Timing Regular/Large-Type 


Cassette/Braille/ Script 


Verbal 1 


30 


45 


75 


Verbal 2 


30 


45 


75 


Verbal 3 


15 


25 


45 


Math 1 


30 


45 


60 


Math 2 


30 


45 


60 


Math 3 


15 


25 


30 
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Table 2 

Demographic Characteristics of Students Testing in Field Trial 
Demographic Variable Percent 



Gender 

Female 32 

Male 58 

Ethnic Group 

American Indian 1 

Asian American 2 

Bladk/ African Amei^ican 11 

Hispanic 4 

White 80 

Other 3 

High School GPA 

A+ 1 

A 2 

A- 5 

B 54 

C 36 

D, E, or F 2 



Note* Total number of examinees = 1,113. 
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Table 3 

Average Verbal and Math Scores for Field Trial Participants 



Group Average 


Verbal 


n 


Average Math 


n 


Spring field trial 
Total 


387 


(108) 


29,369 


443 


(119) 


16,272 


December field trial 
Total 


310 


(99) 


1,110 


341 


(96) 


1,107 


Regular-Type 


315 


(105) 


782 


347 


(101) 


780 


Large-Type 


307 


(87) 


71 


340 


(90) 


71 


Cassette Tape 


303 


(80) 


141 


323 


(74) 


140 


Braille 


294 


(72) 


5 


308 


(59) 


5 


Reader's Script 


292 


(59) 


22 


331 


(92) 


22 



Note* Numbers in parentheses are standard deviations. 
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Table 4 

Completion Rates for Spring and December Field Trials 

for Total Group^ 



Test Section 



Spring Field Trial 



December Field Trial 



Verbal 1 

% completing section 

% completing 75% 

# items reached by 80% 

Total # of items 

n 



87.50 
97.80 

35 
9,995 



89.50 
97.00 

35 
35 
1,055 



Verbal 2 

% completing section 

% completing 75% 

# items reached by 80% 

Total # of items 

n 



85.70 
98.90 

30 
9,995 



88.70 
98.30 
30 
30 
1,055 



Verbal 3 

% completing section 

% completing 75% 

# items reached by 80% 

Total # of items 

n 



71.60 
91.00 

13 
9,995 



89.40 
97-00 
13 
13 
1,055 



Math 1 

% completing section 
% completing 75% 

# items reached by 80% 
Total # of items 

n 

Math 2 

% completing section 
% completing 7 5% 

# items reached by 80% 
Total # of items 

n 



62. 60 
97.50 

25 
9,990 



67-90 
98.00 

25 
9,990 



68.10 
96.60 
24 
25 
1,040 



60.10 
93.30 
23 
25 
1,040 



Math 3 

% completing section 
% completing 75% 
# items reached by 80% 
Total # of items 



93.70 
97.80 

10 
9,990 



97.00 
98.50 
10 
10 
1,040 



^# items reached by 80% not provided for spring field trial. 
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Table 5 

Completion Rates for Spring and December Field Trials 

by Test Format 



Test Section 



Regular Large Braille Cassette Script 



Verbal 1 

% completing section 89.10 81.50 80.00 97.70 86.40 

% completing 75% 96.90 90.80 100.00 98.50 100.00 

# items reached by 80% 35 35 35 35 35 
Total # of items 35 35 35 35 35 
n 740 65 5 130 22 

Verbal 2 

% completing section 89.60 81.50 96.90 80.00 81.80 

% completing 75% 98.40 93.80 100.00 100.00 100.00 

# items reached by 80% 30 26 27 30 30 
Total # of items 30 30 30 30 30 
n 740 65 5 130 22 

Verbal 3 

% completing section 89.50 76.90 100.00 96.20 81.80 

% completing 75% 97.00 89.20 100.00. 99.20 95.50 

# items reached by 80% 13 12 13 13 13 
Total # of items 13 13 13 13 13 
n 740 65 5 130 22 

Math 1 

% completing section 67.00 47.70 60.00 83.10 40.90 

% completing 75% 96.80 90.80 100.00 98.50 90.90 

# items reached by 80% 24 21 23 ' 25 23 
Total # of items 25 25 25 25 25 
n 730 65 5 130 22 

Hath 2 

% completing section 59.70 60.00 60.00 66.20 36.40 

% completing 75% 93.00 98.50 100.00 93.10 86.40 

# items reached by 80% 23 23 23 23 21 
Total # of items 25 25 25 25 25 
n 730 65 5 130 22 



Math 3 

% completing section 97.00 

% completing 75% 98.60 

# items reached by 80% 10 

Total # of items 10 

n 730 



92.30 
96.90 
10 
10 
65 



100.00 
100.00 
10 
10 

5 



100.00 
100.00 
10 
10 
130 



86.40 
90.90 
10 
10 

22 



